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three search strategies in retrieval o f laser articles published in lVo4. 
system provides access to (1) title, (2) author(s), (3) bibliographic reference^, 
literature citations of articles comprising a large part of the body of 
literature. The field of lasers was chosen for study because it is a relatively new field 
and would thus allow a complete view of the scope of its pertinent iterature 
Asburn’s laser bibliography was used as a reference to which articles located in lir 
were compared. There are 25 core journals in the field of physics which are stored 
by TIP. Programmed instructions are used to obtain a response from the system by 
(1) search, defining what literature is to be searched, (2) find, stating item to be 
found, and (3) output, describing requirements of computer output. Three different 
strategies were used to retrieve titles of the 1964 literature. Results of the FO|ect 
indicated that TIP was a reliable means of retrieval of laser articles within the I IK 
library. Similar research will continue in an effort to improve retrieval of scientific 

liferature. (DH) 
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RETRIEVAL OF THE 196A LASER LITERATURE 
USING MIT's PROJECT TIP 

Stella Keenan and Edward Terry 

Articles on lasers published in 1964 were retrieved from the 
mechanized file of physics journals maintained by MIT's Project 
TIP. Three search strategies were used in this study and the 
results compared with the 1964 papers in Edward V. Ashburn^s 
bibliography on lasers published in the Journal of the Optical 
Society of America. The performance characteristics of the TIP 
system are given in this report. 

The Technical Information Program (TIP) at MIT, through the 
facilities of the MAC System, provi des access to the title, author(s) , 
bibliographic references and literature citations of articles published 
in a large volume of the Journal literature of physics. By December 
1965 the holdings of 25 physics Journals, beginning with the first issues 
of 1963 , were available on disc storage. AlP was anxious to encourage 
physicists to use the TIP system, and, through the cooperation of M.M. 
Kessler, director of TIP, and his staff, several studies were designed to 
explore the feasibility of employing TIP as a base for the retrieval and 
evaluation of the physics literature. I t was also decided that AlP should 
record the purposes of its users, describe the search strategies employed 
to locate material in TIP, evaluate TIP performance, and recommend actions 
for TIP improvements. This report presents the results of the first AIP- 
sponsored project with TIP. 

The field of lasers serves as an excellent area for a study of 
this nature, since the recency of the field allows for a complete view 
of the entire scope of the literature. Edward V. Ashburn edits a laser 
bibliography which is published periodically in the Journal of the Optical 
Society of America. Six parts of the bibliography have appeared to date: 
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Part I . 
Part I I . 
Part III. 
Part IV. 
Part V. 
Part VI . 



in Vol . 53, No. 5 
in Vol . 5^ , No . 1 
i n Vol . 55 , No . 6 
in Vol . 55 , No o 8 
in Vol . 58 , No . 2 
i n Vol . 57, No. 1 



p. 6^7 “ 852 
p. 135 - 1^2 
p. 752 - 766 
pJO^O -10^5 
p. 263 - 267 
p. 119 - 1^8 



Western Periodicals Company also published a machine-produced version 
of Ashburn's bibliography. Graph 1 shows the time distribution of 
laser papers as noted by Ashburn in this edition. 



The TIP project undertook to compa re - th e - retr i eva 1 , by various 

search strategies, of this increasingly extensive literature. For the 

purpose of the TIP searches, it was decided to select from Ashburn's 

bibliography the papers which had been published in I 96 A and to compare 

these with the number of I 96 A papers which could be located in TIP. At 

the time of the search, the I 96 A totals for laser materials included 

in Ashburn's bibliography were summarized as follows: 

Articles in journals (See Appendix b) 768 

Articles in books and other non-journal Ik 

material 

Total : 8 A 2 

No. journals publishing laser articles in I 96 A: 13^ 



It should be noted that this total corresponds only to those articles 
found up to the time of the TIP searches (April, 1966). The total in 
Graph I has been up-dated to include all additional materials found up to 
1967 . In fact, A7 additional articles can be accounted for as a result 
of the searches run by TIP. (See Appendix A) 



Because the interest in lasers spans a wide range of fields, 
Ashburn scans the physics, el ectri ca 1 /el ectron 1 c engineering, and medi- 
cal literature for laser articles, as well as a number of journals devoted 
to science in general or miscellaneous specific fields (examples are 



Number 




“ — ^ ^2 i5 Ss 

Year 

PUBLICATION PATTERN OF LASER PAPERS, 1958-1961 
(Number of Papers Plotted Against Year of Publication) 



3 . 
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Nature and J. Amer. Ceram. Soc.). The 25 journals stored in TIP comprise 



the core journals in the field of physics. Since the other types of 
materials are obviously out of TIP scope, the journals in Ashburn's list 
(Appendix B) were separated into several categor i e>.s : 



Tab 1 e 



Distribution of Art i cl es 



in Laser Bibl iography 

No. of No. of 

Articles Journals 



Journals in TIP 
Physics journals not in TIP 
Electri cal /elect roni c engineer! ng 
journal s 

Medical journals 
Other journals 



280 


14 


224 


58 


166 


31 


29 


17 


69 


14 



TOTAL : 768 1 3^ 



The computer operates on the literature by means of programmed 
instructions or commands. To elicit a response from TIP, three such 
commands are used: 

SEARCH -- defines what literature is to be searched 

FIND -- states the items to be found 

OUTPUT -- defines the nature and content of the computer output 
A set of these commands which results in a list of papers matching the 
needs of the user constitutes a search strategy. Three search strategies 
were employed by Kessler in an attempt to retrieve the laser literature 
from the TIP store: 

TIP Search 1 

SEARCH -- the 1964 TIP library 

FIND -- all titles containing one of the following words: 
laser, maser, Brillouin, stimulated emission, or 
coherence. (in a trial run of this search, Raman 
was also tested as a key word. This word generated 
a high recall of papers but a large percentage of 
these were judged to be irrelevant and the term was 
finally deleted from the search). 

OUTPUT -- Print I.D. (journal, volume, page), author(s), and 
title of these papers. 
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The references which were cited by the relevant articles found in this 
search were saved as a separate file labeled file X, Before implement" 
ing Search 2, those cited references which had occurred two or more times 
y^ere arranged by journal, volume and page in order of decreasing frequency. 



TIP Search 2 

SEARCH — the 1964 TIP Library 

FIND — all papers that share common references with those 
in file X (cutting at a citation frequency of 5 in 
file X), less those that have laser, maser, Brillouin, 
stimulated emission, or coherence in the title. 

OUTPUT -- print I.D. (journal, volume, page), author(s) , and 
title of all relevant papers. 

TIP Search 3 

SEARCH — the 1964 TIP library 

FIND — all papers that share common references with the 
remaining papers in file X (cutting at a citation 
frequency of 2 in file X) , less those that have 
laser, maser, Brillouin, stimulated emission, or 
coherence in the title. 

OUTPUT — print I.D. (journal, volume, page), author(s) , and 
title of all relevant papers. 



The results of these searches are contained in Appendix A. It is 
readily seen that TIP was successful in the search for the 1964 laser 
literature in the TIP library. Ashburn's bibliograpny contained 280 
articles from journals included in TIP. The TIP search located 323 
articles, of which 64 were irrelevant. This gives a total of -59 rele~ 
vant articles. Although Ashburn's bibliography had 68 papers that TIP 
failed to locate, TIP retrieved 47 relevant papers which were not listed 

by Ashburn. 
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Taking 3^2 as the total number of laser articles produced in ]S6k 
(see footnote),* 212 or 62 per cent of these were found in common. Addi- 
tional articles found by Ashburn but missed by TIP accounted for 20 per 
cent, while TIP located lA per cent which were not in Ashburn's list. 

About 15 papers (A per cent ) were not retrieved by either method. 

These findings are demonstrated clearly in Figure 1. The total 
number of laser articles found in TIP-covered journals was 327. If one 
assumes that the chances of a paper being missed by either searcher was 
independent of its being missed by the other, then the heavily shaded 
area would represent the possible material which may have been missed 
both by TIP and Ashburn. 

If 3 A 2 is taken as the total number of relevant articles in the 
TIP-stored journals, then Ashburn's search found 280 of these. TIP re- 
trieved 323 articles, 6A of which were judged irrelevant. Although 
Ashburn's bibliography had 68 papers that TIP failed to locate, TIP 
retrieved kj relevant papers which were missed by Ashburn. It is also 
evident that the TIP performance in searches 1 and 2 retrieved the bulk 
of relevant material. Although 2k additional articles were found in 
search 3, ]k of these were judged to be Irrelevant by Ashburn. It was 

* One can estimate the total number of laser papers in the TIP store by 
assuming that the search operations of TIP and of Ashburn are completely 
independent. This is a reasonable assumption since the only point of 
contact between the two was Ashburn's assessment of relevance of the 
material retrieved by TIP. To make the estimate let: 

L = the total number of relevant laser papers produced 
= the search efficiency of TIP 
e' = the search efficiency of Ashburn 

r\ 

then: 259 = ~ 

also: kj = E— (L - 280) ana 68 = E.(L - 259) 

which gives: L = (259) (280)/(212) = 3^2 

and: E^ = 0. 76 and E^ = 0.82 




Precision 



1 





I eric 1 

^ ^ ^ M 
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therefore suggested that this search be eliminated in future studies 
The performance characteristics of the three TIP searches are 



summarized below: 

Table 2. TIP Performance Characteristics 



Search 

1 


No^ Relevant 
Articles Found 

229 


No. Irrelevant 
Articles Found 


Total No. 
Relevant Articles 


Recal 1 


Precis I 


36 


3^2 


0.67 


0.87 


1+2 


2^9 


50 


3^2 


0,73 


0.83 


1+2+3 


259 


6A 


3^2 


0.76 


0.80 


Ashburn 


280 


0 

(by definition) 


3^2 


O.bz 


1 .00 



Ashburn's total recall was 0.82 as compared to 0.67 after TIP Search 1. 

The TIP recall was raised to 0.73 after Search 2 and to 0.76 at the comple- 
tion of all three searches. The results of plotting the precision of each 
search against the recall ratios are shown in graph 2. 



Of the physics material in Ashburn's bibliography, 280 articles 
came from the physics journals listed in TIP. The 58 additional physics 
Journals searched by Ashburn produced only 224 additional articles (See 
Table 1). This means that the 14 productive Journals in TIP were respon- 
sible for 56 per cent of the total articles retrieved from physics 
Journals. Table 1 also shows that the Journals from TIP produced 37 per 
cent of the total material listed by Ashburn in all categories (768 articles). 
The reason for this highly productive performance by the TIP Journals lies 
in the criteria used for the selectivity of materials for TIP input. 



As 

in TIP. 



a result of the work with Ashburn, several changes were made 
It became apparent during Search 2 that the year of publication 
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was necessary for rapid identification of the 1964 papers. This data 
was subsequently added to the TIP bibliographic input. 

In addition, an examination of the 58 physics journals producing 
laser papers which were not in TIP (Appendix B) showed that most of these 
generated 10 articles or less in 1964. The most productive Journals 
not in TIP were Applied Optics which produced 68 articles, the Journal 
of the Optical Society of America which produced 15 articles, and the 
Soviet Journal Optika i Spektroskopiya which had 17. In January 1967» 
Journal of the Optical Society of America was added to the TIP store. 

The results of the project seemed to indicate that TIP was a 
reliable means for locating laser articles within the TIP library, and 
a subsequent study of the 1966 laser literature was planned. A profile 
was also prepared for ASCA (Automatic Subject Citation Alert) for sub- 
mission to the Institute for Scientific Information. This profile was 
set up to explore the effectiveness of retrieval by source author, cited 
author, and source reference. The ASCA profile was submitted so as to 
overlap the TIP retrieval of 1966 material. Reports of these studies 



will be issued separately. 



Laser Searches on TIP Journals ~ 1964 
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APPENDIX B 



JOURNALS IN WHICH LASER ARTICLES HAVE BEEN PUBLISHED IN 1964 



Jou rnal 



Scope 



No. Articles 
In 1964 



Acta Dermato-Venereol , (Sweden) 

Acta Phys. Polon. (Poland) 

Acta Phys. Sinica (China) 

Advances in Phys. (G.B.) 

AIAA Student J. (U.S.A.) 

Akad. Nauk Armianskoi SSR, Dokl . 

Alta Frequenza (Italy) 

Amer.. inst. Aeronaut. J. 

Amer. J. Ophthal . 

Amer. J. Phys. 

Amer. J. Surgery 
Ann. Oculist (France) 

Ann. Phys. (Germany) 

Ann. Surg. (U.S.A.) 

APL Tech. Digest (U.S.A.) 

Appl . Opt. (U.S.A.) 

Appl. Phys. Letters (U.S.A.) 

Arch. Dermatol. (U.S.A.) 

Arch. Ophthal (U.S.A.) ^ /, i \ 

Atti. Fond. Giorgio Contrib. 1st Nazi. Ottica (Italy; 

Bell System Tech. J. (U.S.A.) 

Bri t . J . Appl . Phys . 

Bri t . J . Ophthal . / , 

Bull. Acad. Polon. Sci . Ser. Sci . Tech. (Poland) 

Bull. Soc. Ophthal. France 

Bull. Univ. Osaka Prefecture (Japan) 

Canad. Aeronaut. Space 
Canad. J. Phys. 

Canad. Med. Assoc. J. 

Cancer (U.S.A.) 

Chemi e- I ngen i eur-Tech . (Germany) 

Compt. Rend. Acad. Sci. (France) 

Contemp. Phys. (G.B.) 

Current Sci . (India) 

Czech. J. Phys. 

Dokl. Akad. Nauk. (U.S.S.R.) 

Electronics (U.S.A.) 

Electronics Progress (U.S.A.) 

Elektron Rundschau (Germany) 

Engi<neering (G.B.) 

Ergeb. Exakt. Natur. (Germany) 

Ixper. Tech. Phys. (Germany) 

Federation Proc. (U.S.A.) 

Fiz. Tverd. Tela (u.S.S.R.) 

Frequenz (Germany) 

IBM J. Res. Develop. (U.S.A.) 

IEEE Intern/ Space Electronics Symposium Record 

(U.S.A.) . ^ „ N 

IEEE Proc. Intern. Conv. Milit. Electronics (U.S.A.) 

IEEE Proc. Natl. Aerospace Electron . Conf . (U.S.A.) 
IEEE Proc. Natl. Electron. Conf. (U.S.A.) 



Med 

Phys 

Phys 

Phys 

Phys 

Phys 

El Eng 

Phys 

Med 

Phys 

Med 

Med 

Phys 

Med 

Phys 

Phys 

Phys 

Med 

Med 

Phys 

Phys 

Phys 

Med 

Phys 

Med 

Phys 

Phys 

Phys 

Med 

Med 

Phys 

M'isc 

Phys 

Phys 

Phys 

Phys 



El 


Eng 


El 


Eng 


El 


Eng 


El 


Eng 


Phys 


Phys 


Mis 


>c 


Phys 


El 


Eng 


Phys 


El 


Eng 


El 


Eng 


El 


Eng 


El 


Eng 



2 

2 

4 

1 

1 

1 

4 
1 

2 

1 

1 

1 

1 

1 

1 

68 

95(TIP) 

1 

3 

1 

5 
2 
2 
1 

2 

1 

1 

l(TIP) 

1 

2 

1 

29 

2 

1 

2 

7 
1 
1 
1 
1 
1 
1 
1 

4(TIP) 

1 

8 
1 

7 

1 

5 
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IEEE Trans. Electron. Devices (U.S.A.) 

IEEE Trans. Instr. Meas. (U.S.A.) 

IEEE Trans. Microwave Theory Tech . (U.S .A.) 
IEEE Trans. Milit. Electron (U.S.A.) 

IEEE Trans. Sonics and Ultrasonics (U.S.A.) 
Indian J. Pure and Appl . Phys . 

Indust. Res. (U.S.A.) 

Infrared Phys. (U.S.A.) 

Instrum. Control Syst. (U.S.A.) 

Intern. Sci. Tech. (U.S.A.) 

Japan. J. Appl. Phys. 

J. Amer. Ceram. Soc. 

J. Amer. Med. Assoc. 

J. Appl. Phys. (U.S.A.) 

Chem. Phys . (U.S.A.) 

Electronics Control (G.B.) 

Fac. Engng. Univ. Tokyo (Japan) 

Geophys. Res. (U.S.A.) 

Inst. Elect. Commun. Engrs. (Japan) 

Inst. Navigation (G.B.) 

Invest. Dermatol. (U.S.A.) 

Math. Phys. (U.S.A.) 

Opt. Soc. Amer. 

Phys. (France) 

Phys. Soc. Japan 
Radio Res. Lab (Japan) 

Res. Natl. Bur. Stand. (U.S.A.) 

Res. Radio Sci. (U.S.A.) 

Sci. instrum (G.B.) 

. Surg. Res. (U.S.A.) 

Jemna Mechanika a Optika (Czechoslovakia) 
Kristal lograf iya (U.S.S.R.) 

Life Sci. (U.S.A.) 

Mem Soc. Ingenieurs Civil de France 
Nach r i ch tentech . Z. (Germany) 



J. 

J. 

J. 

J. 

J. 

J. 

J. 

J. 

J. 

J. 

J. 

J. 

J. 

J. 

J. 

J 



Nature (G.B.) 

Ned. Tijdschr. Natuurk (Netherlands) 
Nucleonics (U.S.A.) 

Nuovo CImento (Italy) 

Opt. i Spektroskopiya (U.S.S.R.) 

Oral Surg., Oral Med., Oral Pathol. (U.S.A.) 
Ordnance (U.S.A.) 

Philips Res. Rep. (Netherlands) 

Photographic Sci. and Engr. (U.S.A.) 

Phys. Chem. Glasses (G.B.) 

Phys. Fluids (U.S.A.) 

Phys. Letters (Netherlands) 

Phys. Rev. (U.S.A.) 

Phys. Rev. Letters (U.S.A.) 

Phys. Status Solid! (Germany) 

Phys. Today (U.S.A.) 

Proc. Golden Gate Materials Conf’. (U.S.A.) 
Proc. 1 EEE (U.S .A.) 

Proc. Nat. Acad. Sci. (U.S.A.) 



El 


Eng 


El 


Eng 


El 


Eng 


El 


Eng 


El 


Eng 


Phys 


Misc 


Phys 


El 


Eng 



Misc 

Phys 

Misc 

Med 

Phys 

Phys 

El Eng 

El Eng 

Phys 

El Eng 

El'Eng 

Med 

Phys 

Phys 

Phys 

Phys 

El Eng 

El Eng 

El Eng 

Phys 

Med 

Phys 

Phys 

Misc 

El Eng 

Phys 

Misc 

Phys 

Phys 

Phys 

Phys 

Med 

El Eng 

Phys 

Misc 

Phys 

Phys 

Phys 

Phys 

Phys 

Phys 

Phys 

El Eng 

El Eng 

Mi sc 



2 

1 

1 

1 

1 

1 

1 

1 

1 

2 

11 (T^P) 

1 

3 

57 (TIP) 

3 (TIP) 

1 

1 

2 

1 

1 

2 

1 

15 

2 

2 (TIP) 

1 

2 

1 

k 

1 

2 

1 

3 
1 
1 

2k 

1 

1 

5 (TIP) 
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